Designing and validating a scale for evaluating the sources of unreliability of a high-stakes test

Abstract The idea of sources other than the test-takers’ knowledge leading to different results on high-stakes tests was motif based which present investigation initiated probable unreliability a test. For this purpose, researchers went through thorough literature review with aim identify issues be counted as test, i.e., MA University Entrance Exam English (UEEE) in Iran. First, 17 UEEE test-takers were asked take part semi-structured interview find out their ideas about such sources. outcome thematic coding information from and interviews 57-item Likert scale questionnaire reviewed by three assessment experts, revised accordingly, piloted 57 test-takers, again 55 items remaining. administered 312 Iran, its reliability construct validity checked Cronbach alpha (.89) exploratory factor analysis, respectively. After checking validity, 46 remained loaded four factors named effect (16 items), structure test external concerns (13 administration conditions role proctors (4 items). study might familiarize developers, administrators, teachers, they should aware developing or preparing for like UEEE.